1. Field of the Invention
The present invention relates generally to a luminance gradation correcting method in an image pickup apparatus used for a video camera or the like, and more particularly to a luminance gradation correcting method for reproducing a dynamic image of an object composed of a light object portion and/or a dark object portion at a natural feeling even though a dynamic range of a displaying unit of an image pickup apparatus for luminance is narrow.
2. Description of the Related Art
In a single-plate type color camera, a ratio of the highest reproducible luminance and the lowest reproducible luminance is two or three, so that a dynamic range of the single-plate type color camera is about two or three times. However, because the luminance in an outdoor place is about 50000 lux on a sunny day and the luminance in a room is about 500 lux, so that a dynamic range of about 100 times is required for the single-plate type color camera in cases where a light object portion placed in the sunny outdoor place and a dark object portion placed in a room are simultaneously photographed. To photograph a particular object composed of a light object portion and a dark object portion as a dynamic image by using an image pickup apparatus of a narrow dynamic range, a new charge coupled device (CCD) operated at a transmission speed two times as fast as that in a conventional CCD is provided for the image pickup apparatus, a first picture signal (called a long-time picture signal) is obtained by photographing the particular object at a long exposure time (for example, about 1/60 second which is the same as that in an image pickup apparatus with the conventional CCD), a second picture signal (called a short-time picture signal) is obtained by photographing the particular object at a short exposure time (for example, about 1/1000 second), the long-time picture signal and the short-time picture signal are combined to produce a synthesized picture signal for each field period, and a luminance gradation in the synthesized picture signal is corrected to make a corrected luminance of an image reproduced by the synthesized picture signal correspond to an actual luminance of the particular object and to set the corrected luminance of the reproduced image within the narrow dynamic range of the image pickup apparatus. In this case, an image of the dark object portion is clearly indicated by the long-time picture signal though an image of the light object portion is not clearly indicated by the long-time picture signal, and an image of the light object portion is clearly indicated by the short-time picture signal though an image of the dark object portion is not clearly indicated by the short-time picture signal, Therefore, even though the dynamic range of the image pickup apparatus with the new CCD is narrow, the dark object portion and the light object portion in the particular object can be clearly indicated by the synthesized picture signal in cases where the luminance gradation in the synthesized picture signal is properly corrected.
2.1. Previously Proposed Art
An image pickup apparatus operated according to a conventional luminance gradation correcting method is described with reference to FIG. 1.
FIG. 1 is a block diagram of an image pickup apparatus operated according to a conventional luminance gradation correcting method.
As shown in FIG. 1, an image pickup apparatus 11 is composed of
an image pickup device 12 having M*N pixels (M pixels in the lateral direction and N pixels in the longitudinal direction) for producing a long-time analog picture signal corresponding to a long exposure time and a short-time analog picture signal corresponding to a short exposure time by alternately performing the photographing of a particular object composed of a light object portion and a dark object portion at a long exposure time and the photographing of the particular object at a short exposure time and transmitting the long-time analog picture signal and the short-time analog picture signal in a pair for each field period; PA0 a pre-processing unit 13 for performing a correlated double sampling and a automatic gain control for the long-time analog picture signal and the short-time analog picture signal and amplifying the long-time analog picture signal and the short-time analog picture signal for each field period; PA0 an analog-digital (A/D) converter 14 for converting the long-time analog picture signal and the short-time analog picture signal into a long-time digital picture signal and a short-time digital picture signal for each field period; PA0 a time-axis converter 15 for producing the long-time digital picture signal and the short-time digital picture signal of which transmission time periods (or time axes) agree with each other by alternately storing the long-time digital picture signal and the short-time digital picture signal in a long-time signal memory and a short-time signal memory for each field period and simultaneously reading out the long-time digital picture signal and the short-time digital picture signal from the signal memories; PA0 a luminance level synthesizing unit 16 for combining the long-time digital picture signal and the short-time digital picture signal to produce a synthesized picture signal having a luminance range, luminance values of the pixels in the luminance range being classified into 16 luminance levels L(i) (i=0 to 15), the 16 luminance levels being determined by equally dividing the luminance range into 16 divided luminance ranges, each luminance level representing luminance values of one divided luminance range having the same divided luminance width Lw, and the synthesized picture signal being classified into 16 pieces of histogram data of values H(i) corresponding to the 16 luminance levels; PA0 a histogram data detecting unit 17 for detecting the values H(i) of the pieces of histogram data, the value H(i) of each piece of histogram data indicating the number of pixels having one luminance level, and the number of pixels having one luminance level being called a luminance frequency; PA0 a luminance gradation characteristic producing unit 18, realized by a microcomputer, for normalizing the values H(i) of the pieces of histogram data to produce values Hn1(i) of pieces of normalized histogram data, replacing the values Hn1(i) with values Hn2(i) of pieces of limited histogram data to limit an inclination of a curved line indicated by a series of summed values Hns1(j)=.SIGMA.Hn1(i) (i=0 to j, j=0 to 15), summing up the values Hn2(0), Hn2(1),--and Hn2(j) to produce a plurality of summed values Hns2(j)=.SIGMA.Hn2(i) of pieces of summed histogram data and performing a normalization processing for the pieces of summed histogram data to produce pieces of summed normalized histogram data having values Hnsn(j), an inclination of a curved line indicated by a series of summed values Hnsn(j) of the pieces of summed normalized histogram data denoting a luminance gradation characteristic of a synthesized image indicated by the synthesized picture signal; PA0 a luminance gradation correcting unit 19 for correcting a luminance gradation of the synthesized image indicated by the synthesized picture signal according to the luminance gradation characteristic to emphasize a luminance contrast between a particular luminance level of a high luminance frequency and a group of neighboring luminance levels adjacent to the particular luminance level and producing a luminance gradation corrected picture signal, the luminance gradation corrected picture signal being classified into 16 pieces of gradation corrected histogram data of values H(i) corresponding to 16 gradation corrected luminance levels; PA0 a main picture signal processing unit 20 for processing the luminance gradation corrected picture signal to produce a luminance gradation corrected video signal; PA0 a displaying unit 21 for reproducing a gradation corrected image of the particular object according to the luminance gradation corrected video signal, a ratio of the highest corrected luminance level to the lowest corrected luminance level being within a narrow dynamic range of the displaying unit 21.
In the above configuration, the image pickup device 12 is made of a CCD operated at a transmission speed two times as fast as that in a normal CCD, a particular object portion composed of a light object portion and a dark object portion is photographed by the image pickup device 12 at a long exposure time (1/64 second) to produce a long-time analog picture signal, and the particular object is photographed at a short exposure time (1/1000 second) to produce a short-time analog picture signal. The long-time analog picture signal and the short-time analog picture signal are alternately produced in the image pickup device 12, and the long-time analog picture signal and the short-time analog picture signal are transmitted in a pair for each field period. Thereafter, the long-time analog picture signal and the short-time analog picture signal are processed in the pre-processing unit 13 and are converted into a long-time digital picture signal and a short-time digital picture signal in the analog-digital converter 14. Thereafter, the transmission timing of the long-time digital picture signal is made agree with that of the short-time digital picture signal in the time-axis converter and the long-time digital picture signal and the short-time digital picture signal transmitted at the same time are combined to produce a synthesized picture signal in the luminance level synthesizing unit 16.
FIG. 2 shows a combining method for producing one synthesized picture signal from one long-time digital picture signal and one short-time digital picture signal.
In FIG. 2, an X-axis indicates a luminance value of the long-time digital picture signal for each low luminance pixel and a luminance value of the short-time digital picture signal for each high luminance pixel, and a Y-axis indicates a luminance value of a synthesized picture signal at the corresponding pixel. A low luminance combination level L1 corresponding to the long-time digital picture signal and an upper luminance combination level L2 corresponding to the short-time digital picture signal are set, the long-time digital picture signal is adopted as the synthesized picture signal for pixels having luminance values equal to or lower than the low luminance combination level L1 because an image of a dark object portion can be reproduced well by the long-time digital picture signal even though the dark object portion is photographed at a long exposure time, and the short-time digital picture signal is used for pixels having luminance values equal to or higher than the upper luminance combination level L2 because an image of a light object portion can be reproduced well by the short-time digital picture signal even though the light object portion is photographed at a short exposure time.
Because the short-time digital picture signal is produced at an exposure time shorter than that in the long-time digital picture signal, a luminance value Yshort of the short-time digital picture signal at one pixel is lower than a luminance value Ylong of the long-time digital picture signal at the same pixel, so that an offset value Yoffset is added to a luminance value Yshort of the short-time digital picture signal for each pixel. Therefore, luminance values of the synthesized picture signal for pixels having luminance values equal to or lower than the low combination level L1 are expressed by Ylong, and luminance values of the synthesized picture signal for pixels having luminance values equal to or higher than the upper combination level L2 are expressed by Yshort+Yoffset. A low synthesized luminance level L3 of the synthesized picture signal corresponds to the low luminance combination level L1 of the long-time digital picture signal, and an upper synthesized luminance level L4 of the synthesized picture signal corresponds to the upper luminance combination level L2 of the short-time digital picture signal. The low synthesized luminance level L3 is usually set to half an upper limit luminance value Lmax of the dynamic range of the displaying unit 21.
Also, a gain K linearly changing in a transition luminance range between the levels L1 and L2 is used on condition that the K=1 is satisfied at the low combination level L1 and K=0 is satisfied at the upper combination level L2, and luminance values of the synthesized picture signal for pixels in the transition luminance range are expressed by K*Ylong+(1-K)*(Yshort+Yoffset). Therefore, the influence of the short-time digital picture signal is increased in the synthesized picture signal as the luminance value is increased in the transition luminance range.
Therefore, a luminance value Ymix of the synthesized picture signal is determined for each pixel.
Because the synthesized picture signal has a plurality of luminance values corresponding to the M*N pixels, in cases where the luminance values are equally classified into 16 luminance levels, 16 pieces of histogram data of values H(i) (i=0 to 15) corresponding to the 16 luminance levels can be obtained. In this case, the value H(i) of each piece of histogram data indicates the number of pixels having one luminance level and is called luminance frequency. Also, because the synthesized picture signal for pixels having luminance values equal to or higher than the upper combination level L2 has the luminance values Yshort+Yoffset higher than the luminance values Yshort, the highest luminance level L(15) exceeds an upper limit luminance value of the dynamic range of the displaying unit 21, so that a ratio of the highest luminance level to the lowest luminance level in the synthesized picture signal exceeds a dynamic range of the displaying unit 21.
Thereafter, the values H(i) of the pieces of histogram data are detected by the histogram data detecting unit 17, and a luminance gradation characteristic of a synthesized image indicated by the synthesized picture signal is produced by the luminance gradation characteristic producing unit 18.
FIG. 3 shows a flow chart of a method for producing a luminance gradation characteristic of a synthesized image indicated by the synthesized picture signal.
As shown in FIG. 3, a sum Hall of the values H(i) of the pieces of histogram data is calculated in a step S101. ##EQU1## Thereafter, the pieces of histogram data are normalized to produce values Hn1(i) of pieces of normalized histogram data in a step S102. EQU Hn1(i)=H(n)/Hall
Thereafter, assuming that a plurality of summed values Hns1(j)=.SIGMA.Hn1(i) (j=0 to 15) are produced from the normalized histogram data as a luminance gradation characteristic and a luminance contrast of the synthesized image indicated by the synthesized picture signal is emphasized according to the luminance gradation characteristic, a slight difference between luminance levels of high luminance frequencies is excessively emphasized, so that a signal to noise (S/N) ratio deteriorates or a luminance contrast between luminance levels of low luminance frequencies is lost. To prevent this drawback, an inclination of a curved line indicated by a series of summed values Hns1(j) is limited. That is, in a step S103, in cases where one value Hn1(i) of the normalized histogram data is lower than a low limitation value Cpmin(i), a value Hn2(i) of a piece of limited histogram data corresponding to the value Hn1(i) is set to the low limitation value Cpmin(i). Also, in cases where one value Hn1(i) of the normalized histogram data is higher than an upper limitation value Cpmax(i), a value Hn2(i) of a piece of limited histogram data corresponding to the value Hn1(i) is set to the upper limitation value Cpmax(i). Also, in cases where one value Hn1(i) of the normalized histogram data is ranged between the limit values Cpmin(i) and Cpmax(i), the value Hn1(i) is used as a value Hn2(i) of a piece of limited histogram data.
Thereafter, in a step S104, a sum of the values Hn2(0), Hn1(1),--and Hn1(j) of the pieces of limited histogram data is calculated for each variable value j (j=0 to 15). ##EQU2##
Therefore, a plurality of summed values Hns2(j) of pieces of summed histogram data are obtained. Thereafter, because the highest luminance level L(15) of the synthesized picture signal exceeds an upper limit luminance value of the dynamic range of the displaying unit 21, a normalization processing is performed for the pieces of summed histogram data in a step S105 by multiplying the value of each pieces of summed histogram data by a constant value K1 to make the highest luminance level L(15) of the synthesized picture signal agree with an upper limit luminance value Lmax of the dynamic range of the displaying unit 21. EQU Hnsn(j)=Hns2(j)*K1 EQU K1=Lmax/Hns2(15)
Therefore, a plurality of summed values Hnsn(j) of pieces of summed normalized histogram data are obtained. In this case, an inclination of a curved line indicated by the series of summed values Hnsn(j) of the pieces of summed normalized histogram data denotes a luminance gradation characteristic of a synthesized image indicated by the synthesized picture signal, so that the curved line is called a luminance gradation characteristic line.
Thereafter, in the luminance gradation correcting unit 19, 16 gradation corrected luminance levels Lc(i) corresponding to pieces of gradation corrected histogram data of a luminance gradation corrected picture signal are produced from the pieces of summed normalized histogram data. EQU Lc(i)=Hnsn(i)
In this case, because the inclination of the luminance gradation characteristic line is adjusted by the value K1, the highest gradation corrected luminance level Lc(15) agrees with the upper limit luminance value Lmax of the dynamic range of the displaying unit 21 to set a ratio of the highest gradation corrected luminance level Lc(15) to the lowest gradation corrected luminance level Lc(0) within the dynamic range of the displaying unit 21.
Thereafter, a luminance gradation corrected picture signal, in which each piece of gradation corrected histogram data of the value H(i) corresponds to the gradation corrected luminance level Lc(i), is produced. Therefore, because a gradation corrected luminance level corresponding to a high luminance frequency represents a divided luminance range having a wide luminance width, a luminance contrast between a particular luminance level of a high luminance frequency and a group of neighboring luminance levels adjacent to the particular luminance level is emphasized in a luminance gradation corrected image indicated by the luminance gradation corrected picture signal.
Thereafter, the luminance gradation corrected picture signal is processed in the main picture signal processing unit 20, and a gradation corrected image of the particular object is reproduced in the displaying unit 21 on condition that a ratio of the highest gradation corrected luminance level to the lowest gradation corrected luminance level is within a narrow dynamic range of the displaying unit 21.
Accordingly, a luminance contrast of a particular luminance level, in which a large number of pixels exist, for other luminance levels can be emphasized according to a conventional luminance gradation correcting method. Also, the highest gradation corrected luminance level Lc(15) agrees with an upper limit luminance value of the dynamic range of the displaying unit 21 according to a conventional luminance gradation correcting method to prevent that the luminance values of the luminance gradation corrected picture signal exceeds the narrow dynamic range of the displaying unit 21.
2.2. Problems to be Solved by the Invention:
However, even though a low-luminance object such as a dark object is only photographed by the image pickup apparatus 11 and the highest luminance level of a synthesized picture signal indicating the dark object does not exceed an upper limit luminance value of the dynamic range of the displaying unit 21, the luminance levels of the synthesized picture signal are unnecessarily adjusted to make the highest luminance level of the synthesized picture signal agree with the upper limit luminance value of the dynamic range of the displaying unit 21. Therefore, there is a drawback that the user views a luminance gradation corrected image of the dark object at an unnatural feeling.